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INTRODUCT LON 


our acdles of learning have several major foci. First, how is 
information stored in the human memory system: how is new information in-. 
tegrated with the existing knowledge? Second, what are the mechanisms 
used by the learner to select information from the presents environment 
and - integrate it with his axiacion knowledge structure? Third, how dons 
the tutor model the knowledpe state of the student and use that mode] to 
sutie the tutorial dialog? 

We have recent ly developed the FLOW system, a ‘Itdpurpose facility 
fat acudten of the learning process. Some of the work with the FLOW system 
has been described previously (Norman, Gentner & Stevens, 1974; Norman, 
Gentner & Stevens, in press). This vepore will be concerned primarily 

with the development of an automated tutor which coustructs a simple model 


of the student and uses that model to assist the student when needed. 


Overview of the system 

FLOW is a simple, interactive computer language whose commands are 
designed for string manipulation. Although FLOW is not intended for prac- 
tical computer applications, it shares many important properties with com- 
mon computer languages and is particularly suitable for our studies in 
learning. The section on the FLOW language describes the language in detail. 

The FLOW system is based around a minicomputer. A cathode ray tube 
(CRT) display Sateinal for the student is attached to the minicomputer. The 
experimenter, who is normally in another room, can moniter the student's 
terminal on a television display. The minicomputer is connected to a large 


i(@ computer in which the automated tutor program resides. The experimenter 
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has a terminal connected to the large computer which he can use to inter- 
act with the automated tutor program and, through that program, with the 
student. There are also facilities for recording complete protocols of 

the experimental session. The equipment used in the FLOW system is des- 
cribed in detail in the section on Hardware. 

Over the past few years, a number of different approaches have. 

been taken towards the development of Ocupdceibaues instructional systems. 
These gata can be gécerally classed as ane sf thres evnen, The fieae 
and most common type is based on the sequence: present text, test, and branch. 
The student is given a séction of coke materia aad then tested on that | 
material,usually with a multiple chotee tents Dependide oa the reault of - 
the test, the student can either be given the next text in sequence, or be 
sent back to a previous section, or be side-tracked to a remedial section. _ 
The second type of wate is often called a& generative system, The SCHOLAR 
aysen originally developed by Carbone! 1 (1970) is a fine example of a 
‘generative system, Here the information (texts, questions and answers) are 
hot explicitly stored, but instead there is a database in which information 
about the subject matter to be taught is stored as a semantic network. Gen- 
erative routines in SCHOLAR can access the database to prepare text or ques~ 
tions for the student or to answer questions from the student. A third type 
of computer-based instructional system is the simulation system. In this 
type the computer is programmed to simulate some domain of interest, such 
as a regulated power supply (Brown, Burton & Bell, 1974) The student has 

a number of operations he can perform on the simulation and learns by see- 


ing the results of these operations. 
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Actual ly, Brown's system can also carry on iiiesie dialogs with the 
student. Our FLOW system takes a oimi lar deneoaeh., giving the student an 
interactive domain to experiment with (an implementation of the FLOW 
language) and tutorial assistance, (For examples of related svstems see 
Barr, Beard & Atkinson, 1974; Koffman, Blount, Giikey, Perry & Wel, 1973; 
Goldberg, 1973, ) The atonated tutor in the FLOW system, however, differs 
considerably from previous systems. In some > preliminary experiments, the 
geudent jeatned FLOW primarily from written instructions, entered his pro- 
grams into the ‘computer and modified his programs based on the results, 
Meanwhile a human tutor was. watching over the ‘shoulder of the student — 
and would answer giiestions or interrupt if the student appeared to , be 
having difficulty. The automated tutor is meant to duplicate the func - 
tion of that human tutor. The student. still. juatne primarily from. written 
instruct ions, but the @utotutor tries to follow nie progress and keep 
track of where he is in the instructions by een cortig what he types into 


the computer. The student need not follow the written instruccions exact- 


‘dy, but may move back or skip ahead, The autotutor will try to follow and 


be ready to give appropriate help if the student gets in trouble. 


sie 
THE FLOW LANGUAGE 

Origins 

The FLOW janueage was originally developed by Professor Jeffrey 
Raskin of the. Visuat Arts Department at UCSD, He wanted to teach computer 
programming.to humanities students who often either had little experience — 
with, or actively disliked mathematics and computers. FLOW was conceived. 
as an sat rOgaeEOry computer language that would be non-threatening and 
easy to understand while still pos. ing the basic functions and construc- 
tions of typical computer languages. After first learning FLEW, the stu- 


dent would then go on to a standard .anguage such as BASIC (Raskin, in press). 


Table 1 shows the commands and statements in ane iuplenentation of 
FLOW used in these axperimente,. (After the experiments described in this 
report, we renamed most of the FLOW commands and statements to make them 
more mnemonic. See the section on Subsequent Developments.) 

In our implementation of FLOW, the student's terminal is connected 
to the minicomputer via a full-duplex link. This means that when the stu- 
dent presses a key on his terminal, the character is first sent to the mini- 
computer, examined by the computer and then echoed back for display on the 
terminal, This arrangement allows two interesting features. The first 
feature is called “typing amplification" by Raskin. As soon as the mini- 
computer can unambigucusly recognize a command or statement from the char- 
acters input by the student it supplies the remaining letters of that 


command, Since most FLOW canmands begin with different letters, it is 
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usually Necessary to type in only the first letter of a statement. With 
the commands and statements listed in Table 1, only the underlined letters 
need to he typed in, the computer: vit supply the rest. Second, since the 
computer can examine each character before displaying it on the terminal, 
syntactically incorrect characters can be rejected. Together these two fen- 
tures virtually eliminate typing and syntactic errors, two ma jor scigeay ict 
frustration for Sestnntag programmers. To illustrate how mney WGEhs, SHEDUNE 
& student is entering his program. After Somnfectag a line, the computer 
does’ a carriage return and line feed, provides a new line number and then waits. 
for a student input. “At this point the minicomputer will accent re 
which begins a FLOW conmand or statement or a number (indicating that the ne 
dent wants to change the line number). If the student presses the "P" key 


the word “PRINT” is displayed on the terminal and the minicomputer waits for. 


& Jegal completion of a PRINT statement. (The legal characters at this penne 


are I, *, and R}). If the student then presses rye the miniconputer wil! 


momentarily display a "J" with an audible tone then backspace, erase the mye 


and wait for a i;egal character, An "I" will be accepted as a legal character; 


the minicomputer displays "IT" does a curriage return, linefeed, displays a new 
line number and waits for the next statement. It is not a recommended pro- 
gramming practice, but as & graphic demonstration of these two features, it 

is possible to bang away at the keyboard, and atways produce syntactically 
correct, executable FLOW programs, although, of course, there is no guarantee 


of what they will do. 
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FLOW has a number of advantages as a subject matter for use in stud- 


tes of learning. Along with other computer languages, FLOW involves both 


conceptual and procedural knowledge. It is problem oriented, There are a 
large variety of problems that can be posed to exercise the atudent 's 
developing knowledge. The language itself and the tasks form a small, concise, 
and well defined body of knowledge. The epeeadeval a problem and what con- 
stitutes an acceptable solution are normally fairly clear, al though there may 
- some debate about the qua lity of a particular solution. ‘The students in 


most of our studies were university undergraduates, and it is relatively easy 


to find subjects from that group with little or no experience with computer 


languages. 
FLOW also has some unique advantages as a subject matter, With typing | 
amplification and the rejection of syntactically incorrect characters, two 


major sources of uninteresting errors are eliminated. The errors we are left 


_ with are concerned with the meaning of individual statements and how groups 


of statements interact, Since FLOW is quite a simple language, a lot of in- 
teresting learning takes place within the first hour. The subject matter 
is complex enough, however, to require up to 10 hours to master, This forms 
a convenient time span for experimenta] studies. Subjects usually enjoy 
learning FLOW with its interactive nature and low level of frustration, 
and are highly motivated to learn. 

The last advantage of FLOW derives from the fact that it is primarily 


& computer mediated tusk. This leads naturally to the possibility for 
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computer interaction during the instructional sequence and to 2ase of co‘ tec- 
tion of experimental data, Also since the task is primarily verbal, it is 
@asy to collect detailed protocols of the tutorial interaction and attempts 


at problem solving for later analysis. 
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HARDWARE ; 

Although this report is concerned mainly with the automated euede, 
the FLOW system is designed for a number of different types of experimental 
studies, Figure 1 is a schematic diagram of our equipment. The equipment 
{except for the 86700) is located in two adjoining experimental booths. 

The system is based around a Microdata 810 minicomputer with 24K 
Syren of acacia (the ‘Spplications described here eeguite only 8K bytes). 
The student varie at a Scientific Measuring Systems 1440 CRT terminal con- 
nected ‘to the minicomputer via a full-duplex 1200 baud (120 characters _ 
Sor Gee oe A ey ene from the CRT terminal is connected 
to a video monitor in the experimenter’ s booth duplicating the display 
on the student's terminal, The signal going from the minicomputer to 


the student's terminal which contains all of the information to be dis- 


= played on the terminal including vertical and horizontal spacing also 


-goes to a recording unit. This consists of a sais eonvareins the dig- 


ital signal produced by the minicomputer into a sequence of ‘tones which 
are then recorded on an audio tape recorder. The tape recorder can 
record up to four tracks of information such as verbal comments from 
the student and tutor in addition to the terminal display, Later when 


the tape recording is played back through the modem, the tones are con- 


. verted back into digital signals which faithfully duplicate the experi-~ 


mental session on the CRT terminal. 


The automated tutor oart of our system :.s contained in programs 


residing on the UCSD’s Burroughs 6700 computer, which is connected to 


our minicomputer via two half-duplex 4800 baud lines. (The two haJf- 
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dinias lines can be replaced with a single (aia“aupien. Conneceken when | 
the system will support ful! duplex.) The 86700 computer is also con- 
nected to aTektronix 4010 display terminal in the experimenter's booth 
which is used to program the B6700 and to communicate between exper imen~ 
ter and the automated tutor during tutorial sessions, In addition the 
B6700 prepares erprckoce! of the cutoreal session for later analysis. 
The eApSEABeneer s booth also. contains a keyboard and Teletype terminal 
which are ver to dommindcate with the minicomputer. | 

The minicomputer controls most of the flow of information. It 
monitors ‘the student’ s keyboard, intabscece. and runs his programs, 
and writes on the student's display screen. ‘Input itn the student is 
sumtarized and sent to the automated tutor on the B6700 and all mes- 


sages from the autotutor are relayed through the mand Omputens The 


= major information link outside the niintbomputer is that the experimenter 


can interact directly with the autotutor. 
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THE AUTOMATED TUTOR 


The Printed Instructions 


We decided to base the teaching of FLOK on a set of printed in- 
structions primarily because of ease of preparation, flexibility, and 
convenience of use for the student. Few new educations! Bec unolos tee 


can rival the printed page which is self-paced, portable, easily search- 


ed and inexpensive, Figure 2 shows a typical portion of the instructions. 


The instructions are divided into seven units, each of which in- 
volves three types of tasks: reading, entering practice programs, and 
solving li ae These three types of tasks can be seen in Figure. 2. 


The first part of the figure is explanatory text. Then there isa siete 


. program using the new statement which the student can enter and run, 


Finally. the student is given a programming problem to solve. ‘All the: | 


units have this structure; there are one or more sections of text and 


‘practice programs, and then the student is given a problem to solve at 


the end of the unit. We ask the student to not go on to the next sec- 
tion until he is told to. In fact the autotutor will send him on as 
soon as he solves the problem. The student can go back in the instruc- 
tions whenever he likes but we thought he should not move on to the 
next unit until he had mastered the current unit. We were also afraid 
the autotutor might have difficulty if it thought the student was work- 
ing on Problem 4 when he was actually working on Problem 5, 

New commands and statements are introduced in each section. With 
typing amplification naive students often accidentally enter statements 


which they know nothing about, leading to general confusion. To avoid 


al ts 
this <eub lem, the minicomputer initially isos not respond to any key 
tran the student's keyboard. As the student oval through the in- 
structions, keys are eseivaced at the time that the corresponding state- 
ments are introduced, While the student is working on Unit 1, for 
instance, only the H,P,",R,L,N and £ keys are activated (corresponding 
“to HELP, PRINT ‘string’, RUN, LIST and NEW). When the student success- 
fully domo tetas the problem at the end of unit 1, the autotutor gives 
him a congratulatory message, tells him to go on to Unit 2, and ac- 


tivates the keys needed for that unit. 


Péliwetiy the Student 

During the development of the autotutor, all its functions Gene 
originally carried out. by. a human tutor. Parts of ie: tutoxial fune-.. -- 
"tick were then tated over by the computer until it was compieesty auto. 
‘mated. The human tutor typical ly forms P rather simple aiedel oF the stu- 
dent containing the following information: 
1) where the student is in the instructions, and 2) his current program. 
The human tutor refers to the instructions and general knowledge to 
generate: 3) the student's next sinecees input, and 4) the maximum time 
it should take him to enter the input. If the student entered the expect- 
ed input, the human tutor would advance the student model to the next 
point in the instructions. If the student entered an unexpected input, 
the tutor would use the instructions and general world knowledge to try 
and deduce why the student had done that, and where he now was in the 


instructions. Rather than have the autotutor generate maximum pause 
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lengths and the meaning of dnexeected inputs from the ieseeuetlens 
as a human tutor does, we decided that that would be és difficult a 
task and entered the maximum pause length and a list of inputs explic- 
itly for each stete. 

One of the major tasks of the autotutor is to follow the stu- 
dent. The seudene normally proceeds sequentially through the instruc- 
Sibu, But from time to time he may make errors and decide to repeat | 
things or even go back to an earlier jae of the instructions, A con- 
fident student may skip over some of the practice programs and start 
icmediateiy on tie problem, Through all this, the autotutor monitors  —__ 
what the student is typing into iaexaieal and tries és keep track 
of where he is in the instructions. 

The instructional sequence. is divided into 136 states... During | 


the grechars parts of the instructions, a state - uaually. corranidads to a 


' FLOW statement or ccenaias although in the first unit, states may cor- 


respond to individual characters and partial commands. During the prob- 
lem part of the instructions, each problem corresponds to essentially 

a single state. Each state has associated with it a list of information 
which the autotutor uses to follow the student. These states and their 
associated lists are all that the autotutor knows about the instruc- 
tions. Because the autotutor employs two distinct strategies for the 
practice states and the problem states, we will discuss each of these 


separately, 


Practice 
The autotutor gives three types of messages during the practice 
part of a unit: 1) standard messages in response to a request for help, 


or if the student pauses too long, based on the next expected input 
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(Try typing a 3 2) messages in response to a serious error, 
(usually a NEW out of sequence) which tel! what should have been typed 
and send the student back to a previous section to try again (You should 
have typed ; Please go back to Section __); 3) special eswages 
peculiar to a given state or class of states which clarify usage of cer- 
tain kevs or commands or explain procedures (e.g., If during the first 
unit the siadene types the letter "co" when he: should have typed the num- 
ber "0", he gets the message, "Zero ©) is iocated to the right of 9. 
Don't use the letter 0 for zero.") © 

eo Pitaeebas how two sucoLuber vorke When the atident is in the” 
practice part of the inutruertons; det’ took ta dealt ab cae seckion 
of the instructions in Figure 2. In this unit we introduce the PRINT 


RETURN PRES CNC NE In the practice part we give a brief. explaration of 


= the statement and then a practice program using the new statement which 


the student is expected to enter and run. 


When the student successfully completes the previous problem 


- (Problem 2) the autotutor's model of the student enters State 276. (The 


states are not all numbered consecutively.) When the student then enters 
a NEW command, the minicomputer erases the screen and gives the student 

a line number (010) for his new program. Meanwhile the autotutor advances 
the student model to State 277. We will examine State 277 in detail. 


Associated with State 277 are four possible student inputs and a 


_mMaxinum pause length. The autotutor thinks that the student is about 


to type in the practice program and waits for a student input. The five 


inputs which it is looking for in particular are: 
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PRINT *HELLO' ‘This is what the autotutor expected anc it simply 
advances the student model to the next state 
(State 278). 


PRINT ' BASKET’ This is the first statement for the solution of 
Problem 2, so the autotutor assumes that the stu- 
dent has skipped the practice section and advances 
the student model to the beginning of Problem 2 
(State 282). 


PRINT ‘anything else‘ The autotutor assumes that it just. has a creative 
. student who didn't like the word ‘HELLO and advane- 
es. him to State aves 


NEW The autotutor assumes that the student had made 
, some errors and decided to begin again. The stu- 
dent model is kept in State 277, and to keep some 
control over the chaos, the student is asked to : 
- start at Section 13. 


HELP ‘The autotutor sends che message: TRY TYPING A 
_ 7 -_ PRINT 'HELLO'. The student model remains in 
State 277. 


Any dihoe input is assumed to be an error watch the student will notice and 


correct, The student model is left in State 277. 


The maximum pause length for State 277 is 30 seconds. Failure to press any 
key for 30° seconds is equivalent to typing HELP. The student gets the mes- 
sage "TRY TYPING A PRINT 'HELLO'" and tne student model remains in State 277. 


(The maximum pause lengths, which vary from 30 seconds to 5 minutes are en- 


tered explicitly by the experimenter.) 


“Problems 


When the student finishes the practice part of the instructions, the 
student model is advanced to the state corresponding to the next problem. 
A problem essentially corresponds to a single state; running a correct solu- 
tion to the problem advances the student to the next state. Associated 


with each problem state, the autotutor has a program which is a correct 
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solution of the problem, If the peonecs requires input (in FLOW, that 
wou ld ie in the torm of a TEXT statement), the autotutor will also have 
several test inputs. As the student types in his program, the siniconsak- 
er senile the statements to the autotutor. | 
If the student vind his program, asks for help or pauses too long, 


the autotutor runs the student's program along with the correct program, _ 


pean ene in its test inputs Lf MERE SEREY = if the two Programs gen- 


erate essentially the. same output, the student's program is judged correct, 
and the student is given a congratulatory message and sent on to the next 
unit, The autotutor {s somewhat lenient about certain discrepancies in _ 
the student's output such as extra spaces. Within the rather smal] scope 
of FLOW and the programming problems we use, our procedure of comparing 


the outputs of a correct program and the student's program works quite well. 


‘If the outputs are not essentially the same, the autotutor does a line 


by ‘ine comparison of the programs and informs the student of the first 

discrepancy it finds between the student's seoRcai and the correct program. 
The progran is viewed topologically as groups of non-control state- 

ments bounded by control statements. (Control statements alter the linear 

flow of command of execution; they are STOP, JUMP TO nnn, and IF IT IS 'c! 

JUMP TO nnn.) The non-control parts are examined statement by statement 

as previously described. When a control statement is encountered, 

the branch is taken to the new portion of the program and this is examin- 

ed. This is a recursive process, and if this new portion passes inspec- 

tion, then the statement following the initial contro} statement is examined. 


Thus the whole executable program wil] be canvassed for errors. 


oe 

Example of a Cuctedat Interaction 

To illustrate the action o: the autotutor, we will present a sore 
tion of a tutes interaction ee with the autotutor's interpretation 
of the situation in terms of its modification of the student model. Al- 
though this dialog is taken from actual tutorials, we have blended to- 
gether several tutorials with different students to be able to sey a 
variety of seident errors and. how the autotutor handles. them. As we 


start this excerpt, the student has just correctly solved Protjem 2. 
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Summary of the Automated Tutor 
As the student works trom written instruct ions, the autotutor mon 
itote the student's keyboard inputs and tries to follow the stu- 
dent's progress through a series of states corresponding to the written 


instructions. The states corresponding to the practice parts of the in- 


weducedons have an expected caine length and a list. of possible trues 


associated with them. Corresponding to each possible input is a destina-. 


tion state and possibly a message. In a given state, if the student 


types in one of the inputs on the list, the autotutor prints out the 
message if. any. and advances. the model of. the. student to the. destination oe 
state, If the. student. maPes in an input not on the list, the student 

mode |} remains in the same state. Except for these messages, which are 


rare, che only time that the student is aware of the presence of the 


autotutor is when he asks for help or - pauses for a long time and exceeds 


the expected pause length. Then the autotutor prompts the student with 
the next eeveeted input. 

Somewhat surprisingly it has been our experience that the autotut or 
usually makes fewer errors in following the student through the practice 
parts than a human tutor. Analysis of protocols from early sessions with 
human tutors showed that the human tutors often were quite mistaken in 
their judgments of where the student was in the instructions, The main 
problem seemed to be that crucial key presses were missed due to lack 
of attention, Of course, eternal vigilance is one of the supreme virtues 


of a computer, anc thic seems to give it the edge in this task, 
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The operation of the autotutor while the student is worsing on a 
problem is much different. Here the autotutor only has an example of a 
correct solution to the programming problem, With the aveptten of tuo 
messages which are sometimes given early in the problem, all of the mes- 
sages from the autotutor are Benerated aigortthntcally based ola _ Sompar = 
ison of the student* S program and the correct program, In addition to 
assisting the student with an incorrect program, the autotutor also runs 
the studene’s program whenever the student runs it, and if ths program 
seems correct, the autotutor congratulates the student and asks him to. 


start on the next. nee. aw Gatun ad ae ~ ee Mera n. cee 
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SUBSEQUENT DEVELOPMENTS 


Command Names 


Students learning FLOW often found some of the names of FLOW 


Statements and commands obscure or confusing. In an effort to clarify 


their functions, we have renamed many of the instructions. Table 2 


lists the old instructions and the corresponding new instruction. 

We have also altered due display generated by the WALK command 
ro make it tlearer what is happening: eight lines of the student's pte” 
gtain ‘aré displayed ‘on the upper portion of the screen along with a 


pointer indicating the statement currently being executed, 


‘Conceptual FLOW _ 


One of our original interests was to teach students the FLOW 


. commands and statements and study the spontaneous development. of program 


ming concepts. Thus the written instructions and automated tutor des- 


eribed in this report are oriented around the statements in the FLOW 
language. Concepts such as control transfer and loops are not explicitly 
treated. Some tutorials we have conducted indicate that instruction bas- 
ed on programming concepts rather than the specific statements would be 
more fruitful, especially for students with no previous experience with 
Computers or programming. To produce a FLOW instructional system similar 
to the one discussed here but with a conceptual emphasis would invoive 
rewriting the instructions and modifying the autotutor to analyze student 


programs in tems of the relevant concepts. An interest of ours, in 
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another part of the FIOW study, is the manner in which students form their 
devel oping knowledge intu schemata, These schemata usual ly seem to be con- 
cerned with concepts such as re A vers ion of fnebenatten in FLOW 
oriented towards the programming concepts would thus also connect in more 


directly with our studies of schemata nd how they are used in learning. 
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TABLE 1 
FLOW INSTRUCTIONS 


Commands 
HELP . . Requests help from tutor or autotutor. 
_ NEW - Erases current program. 
RUN oe . Runs current program. 
WALK Steps through the current program. One line of the pro-. 
sie gram is executed each time the space bar is pressed. 
The program statements are displayed as they are execut~ 
ed on the upper part of the screen, variables are dis- 
played in the middle of the screen, and any resulting 
output is displayed in the lower part of the screen, 
This. command is a very useful debugging aia. : *% 
List ; Displays the current program. 


Program Statements | | 
PRINT “character string’ Displays the character string. 


PRINT RETURN Does a carriage return and line feed. 
PRINT IT . . Displays the variable IT. 
JUMP TO nnn Statement number nnn is executed next, (If there is no 


statement with line number nnn, the statement with che 
next highest line number is executed. If all the line 
numbers are lower than.ngn, the program stops.) 


. TEXT IS 'string' Defines a character string to be used as data. 


GET IT Sets the variable IT to successive characters in the TEXT 
a statement. 
if IT IS *x! JUMP TO nnn If the variable IT is the same as the character in quotes, 


the JUMP is performed. 


COMMENT character string No effect. 

MAKE COUNTER ZERO 

- ADD ONE TO COUNTER 

DECREASE COUNTER BY ONE 

| PRINT COUNTER Displays the variable COUNTER. 
AF COUNTER IS tdddd JUMP TO ann 
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TABLE 1 (cont'd.)_ 


Notes: System Commands are executed immediately on entry. Program statements are 


a 
ERIC 


used to write stored programs. 


The COMMENT statement and the ones below it were -ot used in the studies des- 
cribed in this report. 


The programmer types in only the underlined characters, the computer supplies 
the rest. (See the section on Typing Amplification.) 


The programmer must type in the first two letters of the NEW command. This 
is designed to help prevent accidental erasure of programs. _ 


The lower case letters in the statements correspond to slots to be filled in. 
by the programmer. In particular, the phrases “character string" or "string" 
can be replaced by any string of characters and the phrases dddd" and ."nnn'! 
are meant to be replaced by numbers of up to 4 and 3 digits respectively. 


TABLE 2 
NEW FLOW INSTRUCTIONS - 


Old Instructi on Ee 4 - New Instruction 


HELP | HELP 
NEW | | » ERASE 
RUN | | | RUN 
WALK | WALK 
LST LIST ahae et Kerker e4 
- PRINT ‘string’ Ln Soave aad DISPLAY "string". 
PRINT IT | ‘DISPLAY VARIABLE 
PRINT COUNTER DISPLAY COUNTER 
PRINT RETURN : BEGIN ON NEXT LINE 
JUMP TO pon JUMP TO nan 
TEXT IS ‘string’ TEXT IS "string" 
GET IT GET VARIABLE 
IF IT IS 'x' JUMP TO non IF VARIABLE IS "x" JUMP TO onn 
STOP QUIT 
MAKE COUNTER ZERO MAKE COUNTER ZERO 
7 ADD ONE TO COUNTER ADD ONE TO COUNTER 
DECREASE COUNTER BY ONE SUBTRACT ONE FROM COUNTER 


IF COUNTER 1S tdddd JUMP TO non IF COUNTER 1S fddd JUMP TO nnn 
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Student's Booth 


Microphone 


CRT | 
Terminal 


Micro 810 
Minicomputer | 


Terminal 


Experimenter's Booth 


Figure 1 


The FLOW Expe~imental Facility 
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PROBLEM 3: 

Section 12 

You are now going to learn how to make the compute type a vertical 

string of BASKETs, with each word on i HeparAte line. That is, the 

result should look like this: a 
BASKET 


BASKET 
BASKET © 


BASKET 
Section 13 
You should find — an easy program to write. But you "need one 
anew line. We ds that by printing a carriage return: 
PRINT RETURN 
(Note that no quote marks are used for this command.) 
Here's a program using PRINT RETURN: 
010 PRINT ‘HELLO’ 
013 PRINT RETURN 
020 PRINT ‘GOODBYE' 
Type NE. Then type in this program and run it. 
The computer will print: 
RUN 
HELIO 
GOODBYE 
HALT 
030 
Section 14 
Now try Problem 3: Modify your program from Problem 2 (see Sec. 9} to 
make the computer print an endless vertical string of BASKETs, one on 


each line, using the procedures given on Page 3 for modifying your pro- 
gram. 


After you've made your modifications, list your new program to see what 
it looks like. Then run the program to be sure it works. 


Figure 2 


A Portion of the Printed Instructions 


Navy 


& Dr. Marshall J. Farr, Director 
Personnel and Training Research Programs 
Office of Naval Research (Code 458) 
Arlington, VA = 22217 


1 ONR Branch Office 
495 Summer Street 
Boston, MA 02210 
ATTN: Research Psychologist 


1 ONR Branch Office 
1030 East Green Street 
Pasadena, CA 91101 
ATTN: E.E. Gloye 


' 4 ONR Branch Office 


536 South Clark Street. . 
Chicago, TL 60605 . 
ATTN: M.A. Bertin 


6 Director . 
Naval Research Laboratory 
Code 2627 

Washington, DC 20390 


12 Defense Documentation Center 
Cameron Station, Building 5 
5010 Duke Street 

Alexandria, VA 22314 


Special Assistant for Manpower 
OASN (MERA) 

Pentagon, Room 4E794 
Washington, DC 20350 


LCDR Charles J. Theisen, Jr., MSC, USN 
4024 

Naval Air Development Center 
Warminster, PA 18974 


Chief of Naval Reserve 
Code 3055 
New Orleans, LA 70146 


Dr. Lee Miller 

Naval Air Systems Command 
ATR-413E 

Washington, DC 20361 


CAFT John F,. Riley, USN 
Commanding Officer 

U.S. Naval Amphibious School 
Coronado, CA 92155 
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Distribution List 


Seaiee 


‘Behavioral Science rae 


Naval Command & Management Division 
U.S. Naval needy. ; 

Luce Hall 

Annapolis, MD 21402 


Chief of Naval Education & Training 
Naval Air Station 

Pensacola, FL 32508 - 

ATTN: CAPT Bruce Stone, USN 


Mr. Arnold Rubinstein 

Naval Material Command (NAVMAT 03424) 
Room 820, es Plaza #6 
Washington, DC 20360 


Commanding Officer 

Naval Medical Neuropsychiatric 
“Research Unit. 

San Diego, CA 92152 


Director, avy Oscupational Task | 
Analysis Program (NOTAP) 


- Navy Personnel Program Support 


Activity . 
Building 1304, Bolling AFB 
Washington, DG 20336 


Dr. Richard J. Niehaus 

Office of Civilian Manpower Managenent 
Code O6A 

Washington, DC 20390 


Department of the Navy 

Office of Civilian Manpower Management 
Code 263 

Washington, DC 20390 


Chief of Naval Operations (OP-9872) 
Department of the Navy 
Washington, DC 20350 


Superintendent 

Naval Postgraduate School 
Monterey, CA 93940 
ATTN: Library (Code 2124) 


Commander, Navy Recruiting Command 
4015 Wilson Boulcvard 

Arlington, VA 22203 

ATTN: Code 015 


Mr. George N. Graine 
Naval Ship Systems Conmand 
SHIPS 047012 

Washington, DC 20362 


Chief of Naval Technical Training 
Naval Air Station Memphis (75) 
Millington, TN. 38054 

ATIN: Dr. Norman J. Kerr 


Commanding Officer — 
Service. School Command 
U.S. Naval Training Center 
San Diego, CA 92133 
ATTN: Code 3030 


Dr. William L. Maloy - 
Principal Civilian Advisor 

for Education & Training : 
Naval Training Command, Code OLA 
Pensacola, FL 32508 


Dr. Alfred F. Smode, Staff Consultant 
Training Analysis & Evaluation Group 
Naval Training equlleens opal 

Code N-OOT 

Orlando, FL 32813 


Dr. Hanns H. Wolff 

Technical Director (Code N-2) 
Naval. Training Equipment Center 
Orlando, FL 32813 


Chief of Naval Training a 
Code N-21 

Building 45 

Naval Air Station 

Pensacola, FL 32508 


Dr. Robert French 
‘Naval Undersea Center 
San Diego, CA 92132 


CDR Richard L. Martin, USN 
Mghter Squadron 124 
NAS Miramar, CA 92145 


Dr. John bord 
Navy Personnel R&D Center 


San Diego, CA 92152 


Navy Personnel R&D Center 
San Diego, CA 92152 
ATTN: Code 10 


D. M. Gragg, CAPT, MC, USN 

Head, Educational ’ Programs Development 
Department 

Naval Health Sciences Biucation and 
Training Command 

Bethesda, MD 200124 
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Headquettens 


U.S. Army Administration Center 


Personnel Administration Combat 
Development Activity 

ATCP-HRO 

Ft. Benjamin Harrison, IN. 46249 


Armed Forces Staff College 

Norfolk, VA 23512 

ATTN: Library | 

Director of Research | 

U.S. Army Armor Human Research a 
Building 2422 Morade Street 

Fort Knox, KY 40121 

ATTN: Library | 


~ Commandant = =«s = 


United States hea: Infantry § School — 
ATTN: ATSH-DET 
Fort Benning, GA 31905 . 


Deputy Commander 

U.S. Army Institute of Administration 
Fort Benjamin Harrison, IN 46216 — 
ATTN: EA 


Dr. Frank J. Harris : 
U.S. Army Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 


Dr. Raiph Dusek 

U.S. Army Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 


Mr. Edmund F. Fuchs 

UeS. Army Research Institute 
1300 Wilson Boulevard 
Arlington, VA 


Dr. Leon H. Nawrocki 
U.S. Army Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 


Dr. J.E, Uhlaner, Technical Director 
U.S. Arny Research Institute 

1300 Wilson Boulevard 

Arlington, VA 22209 


Dr. Joseph Ward 

U.S. Army Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 


HQ USAREUR & 7th Arny 
ODCSOPS 


ther Government 


‘Dr. Eric McWilliams, Program Manager 
- Yechnology and Systems, TIE — 

National oe Foundation | 
‘Washington, DC 20550 


Dr. Andrew R. Molnar 

Technological Innovations in Education 
- National Science Foundation 
' Washington, DC 20550 | 


. Br. Marshall S. Smith - 

Asst Acting Director 

_ Program on Essential Skills . . 

' National Institute of Education 
Brown Bldg, Rm 815 
19th and M St., NW. 
Washington, D.C. 20208 


asgellaneous 

Dr. Scarvia B. Anderson 
Educational Testing Service 
17 Executive Park Drive, N.E. 
Atlanta, GA 30329 


Dr. John Annett 
_ The Open University 
- Milton Keynes 
Buckinghamshire 
‘ENGLAND | 


_ Dy. Richard C. Atkinson 
_ Stanford University 

. Department of Psychology 
. Stanford, CA 4305 


Mr. Kenneth M. Bromberg 

Manager ~ Washington Operatd ons 
Information Concepts, Inc. 

(1701 North Fort Myer Drive 
Arlington, VA 22209 


Dr. Ronald P. Carver 
School of Education 

University of Missouri--Kansas City 
Kansas City, MO 64110 


Contry Research Corporation 
4113 Lee ead 
“eric 22207 


JA Fuirtoxt Provided by ERIC 


weeS-, 


Dr. Allan M. Collins | 
Bolt Beranek and Newnan, Inc. 


. 50 Moulton Street 


Cambridge, MA 02138 


Dr. H. Peter Dachler 
University of Maryland 


Department of Psychology 
College Park, MD 20742 


Dr. Rene! V. Dawis 
University of Minnesota 
Department of Psychology 
Minneapolis, MN 55455 | 


ERIC 
Processing and Reference Facility 


4833 Rugby Avnue 


Bethesda, MD 2001, 


Dr. Victor Fields. 
Montgomery College 
Department of Payohology 
peektad te; 2? 20850 


Dr. Edwin A. Fleishman 

American Institutes for Research 
Foxhall Square. 

3301 New Mexico Avenue, N.W. 
Washington, DC 20016 


Dr. Robert Glaser, Director 
University of Pittsburgh 

Learning Research & Development Center 
Pittsburgh, PA 15213 


Dr. Henry J. Hamburger 
University of California 
School of Social Sciences 
Irvine, CA 92664 


Dr. M.D. Havron 

Human Sciences Research, Inc. 
7710 Old Spring House Road 
West. Gate Industrial Park 
Mclean, VA 22101 


HumRRO 

Division No. 3 

P.0. Box 5787 

Presidio of Monterey, CA 93940 


Air Force 


1 Research Branch 
AF/DPMYAR 
Randolph AFB, TX 7B148 


1 Dr. GeA. Eckstrand ( AFHRL/AS) 
Wright-Patterson AFB 
Qhie 45433 


1 Dr. Ross L. Morgan (AFHRL/AST) 
Wright-Patterson AFB 
Ohio 45433 


wry 
Colorado” 80230 


1 Major PJ. ‘DeLéo 7 
Instructional aioe Branch 
AF Human Resources Laboratory 
Lowry AFB, CO 80230 


1 AFOSR/NL _ | 
1400 Wilson Boulevard 
| Arlington, VA 22209 
1 Conmandant 
USAF Schoo? of Aerospace = aia 


Aeromedical Library 
Brooks AFB, TX = eee 


1 Dr. Sylvia R. Mayer (MCIT) 

Headquarters Electronic Systems Division 
LG Hanscom Field 
Bedford, MA 01730 


1 CAPT Jack Thorpe, USAF 
Flying Training Division (HRL) 
Williams AFB, AZ 85224 


Coast Guard 

1 Mr. Joseph J. Cowan, Chief . 
Psychological Research Branch (G-P-1/62; 
U.S. Coast Guard Headquarters 
Washington, DC 20590 
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Ma Cor . 


1 Mr. E.A. Dover 
Manpower Measurement Unit (Code MPT: 
Arlington Annex, Room 2413 
Arlington, VA 20380 


1 Commandant of the Marine Corps. 


Headquarters, U.S. Marine Corps 
Code MPTI-20 
Washington, DC 20380 


1 Director, Office cf Manpower Utdiliza 
y rters, Marine Corps (Code MPU 
MCB (Building 200%) 

Quantico, VA 22134 


1 Dr. A.L. Slafkosky 
Scientific Advisor (Code RD-1) 
Headquarters, U.S. Marine Corps 
Washington, DC 20380 — | 


Other DOD 


1 Lt. Col. Henry L. Taylor, USAF 
Military Assistant for Human Resour 
OAD (E&LS) ODDR&E 
Pentagon, Room 3D129 
Washington, D6 20301 


1 Mr. William J. Stormer 
DOD Computer Institute 
Washington Navy Yard, Building 175 
Washington, DC 20374 


1 Col. Austin W. Kihler 
Advanced Research Projects Agency 
Human Resources Research Office 
1400 Wilson Bouleverd 
Arlington, VA 22209 


1 Helga L. Yeich 
Advanced Research Projects Agency 
Manpower Management Office 
1400 Wilson Boulevard 
Arlington, VA 22209 


HumkRO 7. 
Division No, ie mentee 
P.O. Box 2036 
Fort Benning, GA 31905 


HumRRO 

Division No. 5, Air Defense 
P.O, Box 6057 

Fort Bliss, TX 


Dr. Lawrence B. Johnson —— 
Lawrence Johnson & Associates, Inc. 
200 S. Street, N.We, Suite 502 
Washington, DC 20009 


Dr. Milton S. Kati 
MITRE Corporetion 
Westgate Rescarch Center 
McLean, VA 22101 


Dr. Steven W. Keele 
University of Oregon — 
Department of Psychology 
Eugene, OR 97403 ; 


Dr. David Klehr 

_ Carnegie-Mellon University 
Department of Psychology 
Pittsburgh, PA 15213 


i Dr. Robert R. Mackie 

_ Human Factors Research, Inc. 
6780 Cortona Drive 
Santa Barbara Research Park 
Goleta, CA 93017 


| Mr. Brian McNally 
Eiucational Testing Service 
Princeton, NJ 08540 


Mr. A. J. Pesch, President 
Eclectech Associates, Inc. 
P.O, Box 178 

North Stonington, CT 0€359 


Mr. Iuigi Petrullo 
2431 North Edgewood Street 
Arlington, VA 22207 
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Dr. Joseph We Rigney 


University of Southern California 
Behavioral Technology Laboratories 
3717 South Grand 

Los Angeles, CA 90007 


Dr. Leonard L. Rosenbaum, Chairman 
Montgomery College 

Department of ‘Psychology 
Rockville, MD 20850 


Dr. George E. Rowland 


Rowland and Company, Inc. 
P.0. Box 61 
Hetero NJ = 08033. 


Dr. Arthur I. Siegel ; 
Applied Psychological Services: 
404 East Lancaster Avenue 
Wayne, PA 19087 — 


Dr. C. Harold Stone 
1428 Virginia Avenue 
Glendale, CA 91202 


Mr. Dennis J. Sullivan 
725. Benson Way 
Thousand Oaks, CA 91360 


Dr. Benton J. Underwood 
Northwestern University 


Department of Psychology 
Evanston, IL 60201 


Dr. David J. Weiss 
University of Minnesota 
Department of Psychology 
Minneapolis, MN 55455 


Dr. Anita West 

Denver Research Institute 
University of Denver 
Denver, CO 80210 


Dr. Kenneth N. Wexler 
University of California 
School of Social Sciences 
Irvine, CA 92664 


